Concerning the real-time performance of the hierarchical convolutional features for visual tracking, a real-time object tracking algorithm based on sparse convolution features was proposed. By analyzing the characteristics of different convolution layers, the equidistant interval sampling method was adopted to extract the sparse convolutional features of each layer. Then the correlation filter response values of each convolutional feature were combined to estimate the location of target object. Finally, a sparse update strategy was applied to improve the computing speed. Experimental results on benchmark dataset OTB-2015 show that the average distance precision of the proposed algorithm is 82.2%, which is 5.25 percentage points higher than that of the original hierarchical convolutional feature tracking algorithm; furthermore, it has better robustness to appearance changes and occlusions. The average tracking speed of the proposed algorithm is 32.6 frames per second, which is nearly 2 times faster than before, which can achieve real-time performance.